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Abstract 

Background: Evidence indicates that post -stroke rehabilitation improves function, independence and quality of 
life. A key aspect of rehabilitation is the provision of appropriate information and feedback to the learner. 
Advances in information and communications technology (ICQ have allowed for the development of various 
systems to complement stroke rehabilitation that could be used in the home setting. These systems may increase 
the provision of rehabilitation a stroke survivor receives and carries out, as well as providing a learning platform that 
facilitates long-term self-managed rehabilitation and behaviour change. This paper describes the application of an 
innovative evaluative methodology to explore the utilisation of feedback for post-stroke upper-limb rehabilitation in 
the home. 

Methods: Using the principles of realistic evaluation, this study aimed to test and refine intervention theories by 
exploring the complex interactions of contexts, mechanisms and outcomes that arise from technology deployment 
in the home. Methods included focus groups followed by multi-method case studies (n = 5) before, during and after 
the use of computer-based equipment. Data were analysed in relation to the context-mechanism-outcome 
hypotheses case by case. This was followed by a synthesis of the findings to answer the question, 'what works for 
whom and in what circumstances and respects?' 

Results: Data analysis reveals that to achieve desired outcomes through the use of ICT, key elements of computer 
feedback, such as accuracy, measurability, rewarding feedback, adaptability, and knowledge of results feedback, are 
required to trigger the theory-driven mechanisms underpinning the intervention. In addition, the pre-existing 
context and the personal and environmental contexts, such as previous experience of service delivery, personal 
goals, trust in the technology, and social circumstances may also enable or constrain the underpinning 
theory-driven mechanisms. 

Conclusions: Findings suggest that the theory-driven mechanisms underpinning the utilisation of feedback from 
computer-based technology for home-based upper-limb post-stroke rehabilitation are dependent on key elements 
of computer feedback and the personal and environmental context. The identification of these elements may 
therefore inform the development of technology; therapy education and the subsequent adoption of technology 
and a self-management paradigm; long-term self-managed rehabilitation; and importantly, improvements in the 
physical and psychosocial aspects of recovery. 
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Background 

Stroke is a global problem and the worldwide incidence of 
stroke is set to escalate from 153 million to 23 million by 
2030 [1]. In the UK, strokes are the largest single cause of 
disability [2] costing the economy of £8.9 billion a year [3]. 

Evidence indicates that post -stroke rehabilitation im- 
proves function, independence and quality of life providing 
that it is long-term, intense, task-specific, context-specific, 
goal-orientated, variable, environmentally enriched and 
crucially, includes feedback on performance [4-9]. However, 
due to ever increasing service demand combined with fi- 
nancial constraints, needs for stroke rehabilitation cannot 
be met [10]. Innovative interventions and service models 
are recognized as being an essential means of delivering the 
changes that are required to meet the challenges faced by 
healthcare [11]. Solutions that allow rehabilitation to con- 
tinue beyond the acute period thereby improving outcomes, 
place less demand upon services and enables stroke survi- 
vors to be less reliant upon services and able to self-manage 
are essential [9,10,12,13]. 

Recent technological advances have prompted the 
development of Robot- Assisted Movement Therapy [14], 
Virtual Reality Technology [15] and Inertial Tracking 
Devices to assist with stroke rehabilitation [16-18]. These 
devices have the potential to provide consistent, detailed, 
individually adapted feedback in the absence of the ther- 
apist [19]. Furthermore, such systems may also have the 
potential to promote self-managed rehabilitation in the 
home, over the longer term [20]. 

However, the drive towards the use of new technologies 
and the promotion of self-management leads to further 
questions regarding the reliance a stroke survivor may have 
upon a therapist for both, motor learning skills [21] and the 
self-management of the resultant long-term disability [13]. 

The Self-Management Supported by Assistive, Rehabilitation 
and Telecare Technologies (SMART) programme 

The EPSRC funded SMART Rehabilitation research 
programme (www.thesmartconsortium.org) [22], began 
in 2003 to develop and test a prototype telerehabilitation 
device (The SMART Rehabilitation Technology System) 
for therapeutically prescribed stroke rehabilitation for 
the upper limb. The aim was to enable the user to adopt 
theories and principles underpinning post-stroke rehabili- 
tation and self-management. This included the develop- 
ment and initial testing of the SMART Rehabilitation 
Technology System (SMART 1) [16,17] and then from 
2007, a Personalised Self-Management System for chronic 
disease (SMART 2) [23,24]. 

The SMART rehabilitation technology system 

The SMART system incorporates a wireless sensor system 
that allows for three dimensional (3D) real-time computer 
feedback whereby the user is able to view the avatar from 



various planes of view [17]. The system uses inertial 
tracking devices worn on the upper arm, wrist and chest to 
record kinematic data from the users when they are 
undertaking the prescribed exercise [24]. Figure 1 illustrates 
a previous participant using the system. 

The SMART system monitors and tracks the upper 
arm rehabilitation movements in real time. It allows 
motion patterns to be identified, analysed and cor- 
rected by both the patient (during and after use), and 
the therapist (on review) [25]. The program is designed 
to enable recording and playback using an avatar pres- 
entation (Figure 2a, b) and provides qualitative know- 
ledge of performance describing the characteristics of 
performance (i.e. you lifted your arm higher that time) 
(Figure 2a and b), and knowledge of results describing the 
result of a performance (i.e. you scored 88%) (Figure 2c) 
and summary feedback (feedback provided over a period 
of time) (Figure 2d) in chart. This feedback allows the 
user to analyse and record the movements performed in 
the absence of a therapist [25]. Crucially, in order to en- 
courage self-appraisal the playback facility allows the user 
to analyse and compare their movement to a reference fig- 
ure displaying the correct movement pattern (Figure 2b). 

Initial findings from technology deployment revealed 
how a number of usability and contextual issues im- 
pacted on the theoretical outcomes underpinning this 
complex intervention [17]. Further investigation was re- 
quired to explore how utilising computer feedback for 
upper limb stroke rehabilitation in the home setting can 
impact on the underpinning theory-driven mechanisms 
and theoretical outcomes. 

This paper describes the application of an innovative 
evaluative methodology to explore the utilisation of 
feedback for post-stroke upper-limb rehabilitation in 
the home [26]. This includes refining the underpinning 
theories embedded within the SMART system. 




Figure 1 User participation with The SMART Rehabilitation 
Technology System in the home-setting (Mountain et al. [17]). 



Parker et al. BMC Medical Informatics and Decision Making 2014, 14:46 
http://www.biomedcentral.com/1472-6947/14/46 



Page 3 of 13 




a: On-screen 
display during exercise. 



b: On-screen 
display following 
exercise. 




Qualitative chart 
feedback. 



d: Summary 
feedback. 



Figure 2 Feedback Screens (a) Concurrent knowledge of performance feedback (b) Terminal knowledge of performance feedback 
(c) Knowledge of results feedback (d) Summary feedback. 



Methods 

Realistic evaluation (RE) explores how a mechanism may 
lead to differences in outcome if it is added to a different 
context: Context + Mechanism = Outcome [27]. Figure 3 
illustrates the RE process and framework used in this 
research. 
Definitions: 

- Context - The conditions that enable or constrain 
the mechanisms. 

- Mechanism - What is it about intervention A that 
produces B? 

- Outcome(s) - The intended or unintended 
consequences of interventions [27]. 

Using the principles of RE [28], focus groups with 
Community Stroke Teams and preliminary testing of the 
SMART system in participants homes were carried out 
to validate and refine hypothesised context-mechanism- 
outcome configurations (CMOCs). This was followed 
by systematically testing the refined CMOCs through 
multi-method case studies using the SMART system for a 
prolonged period (up to five weeks). Data were analysed in 
relation to the hypotheses case by case. This was followed 
by a synthesis of the findings to answer the question, c what 
works for whom and in what circumstances and respects?' 



Theoretical framework and outcomes 

Technology development took account of the theories 
underpinning motor re-learning, self-management and 
behaviour change. To achieve this, the theory-driven 
mechanisms such as, how receiving feedback from 
computer-based technology might encourage independ- 
ent, self-evaluation and self-monitoring of recovery, were 
included to encourage users to carry out intense, repeti- 
tive, self-directed rehabilitation based on goal setting and 
actively problem-solving [29-31]. To achieve optimum 
outcomes, these mechanisms had to be able to support 
this behaviour over time. 

The generation of context-mechanism-outcome 
configurations (CMOCs) 

The process of generating CMOCs involves explor- 
ing the theoretical constructs underpinning the inter- 
vention. These lead to propositions which are then 
set out as CMOCs. Figure 4 illustrates the process 
of moving from theory to the generation of CMOCs 
(Figure 4). 

The theoretical 'ingredients' that are drawn from the 
theories; how the feedback from the SMART system 
incorporates these ingredients are set out in Table 1. 
The resultant propositions are then set as mechanisms 
to describe the process of how users of the SMART 
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Theory 

•Context 

•Mechanisms 

•Outcomes 



Programme 
Specification 

• What works for whom in 
what circumstances and 
respects? 

V y 



Hypotheses 

• What might work for 
whom in what 
circumstances?' 



Observation 

• Multiple methods 



Figure 3 The RE framework in this research (Pawson and Tilley [27] p.58). 



system act upon the intervention stratagem to make 
them work. These are then set as CMOCs. The final 
part of developing the CMOCs involves describing 
possible outcomes. An example of two of the CMOCs 
proposed is presented in Table 2. 

CMOCs 

The CMOCs are set out as: Context (C) + Mechanism 
(M) = Outcomes (O) - what' (M) works 'for whom' and 
in what circumstances' (C) to produce what possible 
outcomes (O). How these outcomes are measured is also 
presented in Table 2. 



The CMOCs are then tested and refined which en- 
ables the researcher to evaluate 'what works for whom 
and in what circumstances and respects?' 

Validating and exploring existing and New CMOCs 

The next process within the RE cycle (Figure 3) involves 
testing and validating the embryonic hypotheses by gain- 
ing the perspectives of those who are deemed important 
stakeholders in the delivery of the intervention. This 
enables the researcher to gain an empirical insight and 
explore existing and new propositions. In other words, 
'this is my theory, what is yours?' [28]. 



yCMOC'S 

Propositions 



/Ingredients 



7 



-Underpinning 
Theories 



/ 



^Yheoretcial 
areas 



Figure 4 The generation of CMOCs. 
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Table 1 The incorporation of theory-driven ingredients within the SMART system 



Theory Topic 



Theories 



Ingredients How the feedback from the SMART system 

incorporates these 



Theory Topic 1 : The theoretical 
approach to post-stroke 
rehabilitation incorporated 
within the SMART system. 



Underpinning Theories: 

Neuroplasticity; 

Motor-learning 



Theory Topic 2: Feedback in 
Post-Stroke Rehabilitation. 



Underpinning Theories: 
Motivation; Operant 
Conditioning; Motor-learning 



Theoretical Outcome: Behaviour 
change and Self-management. 



Underpinning Theories: 
Self-regulation, Social 
cognition, and goal-setting 
theories; Self-efficacy. 



Independent practice 
Intensity 
Problem solving 

Goal setting 
Specificity 
Repetition 
Salience 
Motor learning 

Feedback content 
Feedback schedule 
Rewards 

Goal-setting 

Action planning 
Self-monitoring 
Reinforcement 

Self-management 



Self-efficacy 



Used in the absence of a therapist. 

Increased rehabilitation activity. 

Self-monitoring, self-interpretation, overcoming 
problems encountered in the absence of a therapist. 

Choosing which exercises to perform. 

Matched movement patterns. 

Increased rehabilitation activity. 

Relevant (meaningful) feedback. 

The SMART system provides an opportunity to learn 
implicitly and explicitly through trial and error and 
explicit feedback. 

The SMART system provides KP, KR, verbal and visual 
feedback. It also provides prescriptive feedback (in part). 

The SMART system provides feedback concurrently, 
terminally, after each performance and in summary. 

The SMART system provides the user with the rewards 
of good performance through a red, amber, green chart 
and through scores (depending on exercise). 

The user is able to set specific, measurable, realistic, 
and time specified goals (targets) to achieve that are 
confirmed by the computer feedback. 

The user can choose when to use the system and 
how many sets/repetitions they do. 

The user is able to monitor performance(s) 
independently. 

The SMART system provides the user with positive 
feedback (depending on performance). Others are 
able to observe results. 

The SMART system provides the user with an 
opportunity to problem-solve, make decisions, 
utilise resources, collaborate with others, and take 
action depending on their interpretation of the 
feedback provided. 

The SMART system provides the user with an 
opportunity to evaluate achievement(s), observe 
demonstrations (the avatar), interpret performance(s) 
and changes in physical and emotional feelings as a 
result of usage, and receive feedback which may 
include verbal persuasion from significant others. 



CMOC refinement through stakeholder involvement 
Professional perspectives of the delivery of feedback in 
clinical practice 

Separate focus groups [32,33] with Occupational ther- 
apists (OT) and Physiotherapists (PT) (n = 14) that 
were specifically involved in facilitating physical re- 
habilitation to stroke survivors in the patients' home 
were convened to: 

• Explore the pre-existing context and mechanisms 
underpinning the delivery of feedback during 
community post-stroke rehabilitation to validate the 
embryonic hypotheses. 



• Establish the empirical theory underpinning what 
current practice and provision of information and 
extrinsic feedback therapists employ to assist 
patients to continue with their rehabilitation post 
hospital discharge. 

The following Tables 3 and 4 detail the demographics 
of the therapists. 

The focus groups highlighted how therapists control 
the rehabilitation process; what they include and who 
they include. For example, the therapists appear to make 
clinical decisions based on their empirical knowledge 
and understanding, their evaluation of contextual factors 
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Table 2 Two examples of the CMOCs 



CMOCs Plausible mechanisms: 'what' 



Contexts: 'for whom' and 'in 
what circumstances' 



Possible outcomes 



Measures 



CMOC 1 M1 : Receiving feedback from the 
system might improve the user's 
confidence by confirming performance. 



CMOC 2 M2: By receiving feedback, users might 
feel confident to be able to interpret 
their performance and changing their 
movements to improve subsequent 
performance(s). 



C1: A system that is accessible 



01 : Adoption and development of Observation of use 



(in the home setting) and used by a self-management approach to and avatar replays. 



the stroke survivor, independently 
of the therapist. 



C2: A system that can be used 
independently by the stroke 
survivor in the home. 



rehabilitation (behaviour change). 

- Independent rehabilitation, 
self-evaluation and self-monitoring 
of recovery. 



02: Development of 
self-management skills. 

- Problem-solving. 



User diary. 

Usage of the system. 

Interview data. 

Observation of use 
and avatar replays. 

Interview data. 



and what they believe will be effective for that individual 
patient. It was also suggested that the provision of 
feedback must be adaptable and personalised towards 
the specific personal and environmental context of the 
recipient. Therefore, in order for the mechanisms to 
work, the context must include a system that is adaptable 
and can be personalised for the user. 

Preliminary user testing 

Following the acquisition of all necessary institutional 
ethical and governance approvals, two participants who 
had experience of testing the system and their co-resident 
carers were recruited to the study for the purposes of fur- 
ther refining the CMOC s. They agreed to have the technol- 
ogy installed in their house and were interviewed before 
installation and after it had been in their home for around 
three days. Semi-structured interviews [34,35] were audio- 
taped to ensure the transcriptions were presented verbatim. 
In addition, observations [36,37] and field notes were taken 
to account for informal discussion and physical behaviour. 

This initial testing revealed that firstly, a stroke survivor 
may adopt a self-managed approach to rehabilitation if they 

Table 3 Community stroke team one demographics 



Therapist 


OT/PT 


Years 
qualified 


Qualification 


Stroke speciality 
(time working in 
stroke in years) 


HPA 


OT 


20 


Dip Cot 


10 


HPB 


PT 


10 


Dip Grad Phys 


7 


HPC 


PT 


6 


BSc (Hons) 


2 


HPD 


PT 


6 


BSc (Hons) 


1 


HPE 


PT 


5 


BSc (Hons) 


1 


HPF 


OT 


11 


BSc (Hons); MSc OT 


8 


HPG 


PT 


8 


BSc (Hons); MSc 
Mod asic Bobath 


6 


HPH 


OT 


6 


BSc (Hons); 
Previous BSc (Hons) 


2 


HPI 


OT 


1 


BSc (Hons) 


1 



do not believe that they may become more socially isolated 
as a result of carrying out rehabilitation independently 
(i.e. receiving less help from others); and secondly, their 
motivation to set goals towards recovery is subject to 
having a continued desire to recover [19]. 

The interview data also suggested that the carers were 
able to engage in the rehabilitation process because they 
did not feel that when they provided feedback to the stroke 
survivor, they were being critical of the person they care for 
[19]. This was because the SMART system provides feed- 
back instead of the carer. This allowed for a new CMOC 
hypothesis to be generated and subsequently tested in that 
providing feedback through technology will enable signifi- 
cant others to take a more active role in the rehabilitation 
process and reinforce behaviour/ 

Systematic testing of the CMOCs 

The multi-method collective case-study approach using 
multiple qualitative and quantitative observations enables 
exploration of how the same mechanisms play out in dif- 
ferent contexts, producing different outcomes [27,38,39] . 

Sampling 

For both the preliminary and systematic testing, purposive 
sampling was used to identify participants who had recently 

Table 4 Community stroke team two demographics 



Therapist OT/PT Years Qualification 
qualified 



Stroke speciality 
(time working in 
stroke in years) 



HPI 

HP2 

HP3 
HP4 

HP5 



PT 

OT 

OT 
PT 

OT 



20 

2 
13 



BSc (Hons); 
Basic Bobath 

Dip Cot; MSc 
Mods Basic Bobath 

BSc (Hons) 

BSc (Hons); Basic 
Bobath; Adv 
Bobath; MSc Mods 

Dip Cot 



2 
15 



14 
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been discharged from a period of multidisciplinary com- 
munity stroke rehabilitation and also met the inclusion/ 
exclusion criteria. The researcher was assisted in the 
identification of potential participants by the members 
of the Community Stroke Teams. All five participants 
approached for the study by the therapist(s) were re- 
cruited. Table 5 describes the participant demographics 
(pseudonyms are used). 

Selection criteria 

• Inclusion: 

o A definite diagnosis of stroke (as reported 
by the therapist(s) through medical records) 
and have not been referred for further 
rehabilitation 

and/or receiving further rehabilitation, 
o Able to give informed consent to participation 

in the study and for their G.P. to be informed of 

their involvement 
o Living with co-resident carer 

• Exclusion: 

o Unable to speak or comprehend written English, 
o Presence of aphasia (Frenchay Aphasia 

Screening Test (FAST) score of < 24) [40]. 
o Severe communication, perceptual or cognitive 

disorders (Mini Mental State Examination 

(MMSE) score of < 17) [41]. 
o Visual impairment, 
o Apraxic. 

o Shoulder subluxation or arm pain. 

o Medically unstable and other neurological, 
neuromuscular, or orthopaedic disorders that 
would interfere with task performance. 



The following table describes the participants involved 
in this study. 

Research setting 

Following screening of the volunteers with stroke using 
FAST and MMSE informed consent was obtained 
firstly from the person with stroke and then from their 
co-resident carer. All researcher contact with participants 
and their carers was in the participants' home. These 
allowed for interviews and observations to be conducted 
in a natural environment as well as meet the aims of the 
study; home based rehabilitation using technology. 

Procedures 

Following acquisition of all necessary National Research 
Ethics approval from the South Yorkshire Research Ethics 
Committee (Ref: 08/H1310/63) and governance approval 
from the Sheffield Health and Social Research Consortium, 
the SMART system was provided for individual participants 
to use for up to five weeks in their homes (from initial face- 
to-face contact to removal of the system). Following con- 
sent, participants and carers were both instructed in how to 
use the system and were also provided with a hard copy user 
manual. The people with stroke was then invited to under- 
take upper-limb rehabilitation using the SMART system as 
frequently as they wished during the five weeks and were 
visited for data collection and system support purposes. 

Each participant was initially advised to undertake reach 
forward and reach sideways exercises. This was because they 
were simple exercises that enabled the user to see the avatar 
move in different planes. Depending on observed and re- 
ported ability (and limitations) to perform the exercises and 
the participants expressed desire to try more exercises, they 
were then given others to attempt such as, hand-to-mouth; 
and a catch the ball' game. During and following prescribed 
activity; computer feedback was provided via real-time 



Table 5 Demographics of the participants (pseudonyms are used) 

Case study Age CV A/Side Time since MMSE score/30 FAST score/30 Computer Active range of movement 

affected stroke experience (affected shoulder) 

Mr Brown 70 L/R Hemi 6 months 25 28 + (minimal) 90° Flexion 

45° Abd 

Mrs Green 79 L/R Hemi 8 months 23 25 - (none) 70° Flexion 

70° Abd 

Mr Gray 62 R/L Hemi 5 months 30 30 ++ (moderate) 30° Flexion 

20° Abd 

Mr Blackwell 65 L/R Hemi 5 months 30 29 +++ (extensive) 20° Flexion 

20° Abd 

Mr Redmond 79 L/R Hemi 5 months 27 25 + (minimal) 90° Flexion 

90° Abd 
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3-D images; a qualitative chart and a graph (Figure 2a, d) 
displayed on a lap-top computer. 

The participants were not advised as to the frequency of 
usage or the number of repetitions to perform but were 
encouraged to use the equipment as often as they wished 
which were then logged by the SMART system. This 
allowed the researcher to evaluate the number of sessions 
the participant chose to do. This provided some insight of 
the participants willingness to use the equipment. How- 
ever, they may have felt obliged to use the equipment by 
agreeing to participate in the study. 

Data collection and methods used 

The case studies involved a number of methods of data 
collection before during and after technology use to ex- 
plore the context of the participants; observe the mecha- 
nisms involved in using the technology; and explore the 
outcomes (Figure 5) [32-37,42-44]. 

Data analysis 

Data analysis was based on both the exploration of the 
pre-existing context of the participant and the development 
and refinement of the hypothesised CMOC's using the- 
matic and framework analysis [45,46]. This innovative 
approach to the analysis draws on Yin [39], Miles and 
Huberman [46] and Patton [47] and is underpinned by the 
principles of Pawson and Tilleys RE [27]. This approach 
allowed for themes to emerge from multiple sources of data 
and examines interconnections and relationships between 
the mechanism(s) and context(s) in relation to proposed 
outcomes [48,49], 

Results 

The next stage of the RE cycle (Figure 3) involves the 
specification phase where the findings are synthesised and 
presented as refined CMOC configurations to answer the 
question: what works for whom and in what circumstances 
and respects?' [27,28]. 



'What work works for whom and in what circumstances 
and respects' 

Data analysis reveals that in order to achieve desired 
outcomes through the use of computer technology, key 
elements of computer-feedback such as; accurate, meas- 
urable, rewarding, adaptable, and knowledge of results 
feedback are required to trigger the mechanisms under- 
pinning the intervention. In addition, the pre-existing 
context and the personal and environmental contexts such 
as; previous experiences of service delivery, personal goals, 
trust in technology, social circumstances and practicalities 
may also enable or constrain the underpinning theory- 
driven mechanisms. 

Throughout the observational phase, a number of specific 
theoretical components were found to be of particular im- 
portance including; receiving feedback that was perceived 
as being rewarding, accurate, and included the results of 
performance (knowledge of results feedback) that was 
measurable. These components enabled key mechanisms 
to work such as motivating the users (and carers) to use 
the equipment and having the confidence to continue using 
it. This also impacted on their engagement with the system 
and frequency of usage which in turn served to reinforce 
self-managed rehabilitation. 

Adaptable and personalised feedback 

In order to improve the users' ability to relate to the ava- 
tar, it was suggested that the graphical interface needs to 
be individualised to the user. For example, the user may 
wish to alter the avatar image to look like them (i.e. male/ 
female) and therefore, make it both recognisable and eas- 
ier to relate to. One user found it difficult to relate to the 
avatar as she thought it did not represent her, 

"It's not me on the screen, that's a man and I'm a 
woman!" (Mrs Green) 

In addition, one participant explained that the feed- 
back would be more meaningful if his results were 




•Therapist's and 

researcher's 

Profile: age etc. 
•MMSE, FAST 

screening scores 
•Participant 

Interview 1 
•Documents (ex 

sheets etc.) 
•Carer interview 1 
•NEADL 



Figure 5 Data collection methods before, during and after using the SMART system. 




After using the 
SMART system 



•Observation 

•Observer as 
participant 
•Field notes 
•Diary 

•Analysis of video 
replays 



•Interview 2 with 

participant 
•Interview 2 with 

carer 

•Cross-check 
•Carers 
•Diary 

•Video replays 
•NEADL 
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something he could relate to in everyday life like being 
able to play his guitar or using his hand to hold a 
plectrum, 

"The feedback would be better if I could relate it to 
playing my guitar or holding the plectrum because these 
are things I want to be able to do... goals" Mr. Blacl<well. 

This suggests that further developments in feedback 
should be capable of being tailored towards the user s spe- 
cific goals to enable the feedback to become meaningful' 
to the user. 

Rewarding feedback 

The provision of rewarding feedback impacted on the 
participants motivation to use the system. Three people 
with stroke specifically described how obtaining positive 
results affected their motivation to use the system. 
This was confirmed by Mrs. Greens carer who sug- 
gested that once she had managed to achieve good re- 
sults, she was determined to maintain her performance 
which resulted in her wanting to use the system more 
often (increased usage). 

"It was only when she got all the dots in the green she 
really wanted to go on it again" (Mrs Green's carer). 

Accuracy and reliability 

Whilst positive, rewarding feedback motivated the users 
this was offset by a lack of overall reliability of the tech- 
nology. For example, one of the users described that if 
he observed differences in the on-screen feedback com- 
pared with how he perceived he had performed, he would 
lose trust in the technology, 

"I was expecting better results than that... I thought I 
had done well but that [the computer] says I haven't... 
It's disappointing; it must be the sensors" Mr Gray. 

Knowledge of results and measurable feedback 

All of the users were uninterested in watching their replays 
using the avatar (knowledge of performance feedback). One 
participant explained how the qualitative knowledge of 
results chart was easier to understand and enabled her 
to interpret how well she had performed, 

"I didn't really grasp the two men on the screen. Those 
dots were much easier but it still took a few go's to get 
used to it. I knew if I got in the green Vd done well" 
Mrs. Green. 

Explicit knowledge of results such as, the provision of 
scores with the catch the ball' game (Mr Redmond) also 
increased the determination of the user to improve, 



"It's a good idea; you can see a target to aim for! I can 
see the dots getting higher and my score going up on 
that game" Mr. Redmond. 

Mr Redmonds carer suggested that providing a score 
also enabled him to tell others about his progress, leading 
to him receiving praise from others, which reinforced his 
engagement with rehabilitation. 

"It is easier for him to tell other people that I did 
catch the ball and I got 3100' or... Tm better this week 
I got 3600' and even though it might not make much 
sense to other people, they can tell it is going up and it 
always seems to appeal to him and he wants to do 
better on it each time" Mr Redmond's carer. 

Contextual factors 

Contextual factors also influenced the activation of 
mechanisms. These included; the pre-existing context, 
the personal and environmental context, and the influ- 
ence of the carer and researcher. These are described 
below. 

Pre-existing context - Previous experience of rehabilitation 
(i.e. therapist led rehabilitation to patient led rehabilitation) 
could influence expectations of service delivery. One par- 
ticipant did not believe it was his responsibility to lead his 
rehabilitation, 

"It's not up to me to sit here and do it all myself is it... 
You're the experts you should be doing it" Mr Brown. 

Personal and Environmental context - This included 
personal preferences, stroke severity, acceptance of tech- 
nology and motivation to improve and carry out rehabili- 
tation independently which the research confirmed all 
impacted on the utilisation of feedback from the technol- 
ogy. In addition, the environmental restrictions of space 
and ferromagnetic interference also detrimentally influ- 
enced the use of the SMART system. For example, not 
having somewhere to store the SMART system when not 
in use and the sensitivity of the inertial sensors to large 
metal objects (i.e. radiators) resulted in the SMART system 
being stowed away upstairs. In other words, if the sensors 
were kept near a large metal object, this would affect the 
accuracy of the sensors and distort the on-screen image. 
Because of this moving the system from its place of use to 
its place of storage increased the amount of setting up time 
and effort required prior to use. 

Carer and researcher influence - For the carer, issues 
such as personal skills, feeling empowered to be involved, 
accepting technology in the home and as part of a 
therapeutic package were also key to the utilisation of 
the SMART system. For the researcher, the provision 
of adequate training and support was essential. 
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Quantitative outcomes: the Nottingham Extended 
Activities of Daily Living (NEADL) scores 

Analysis of the NEADL scores [42-44] for each participant 
revealed that irrespective of the number of attempts to 
use the SMART system or the number of sets and repeti- 
tions, the participants did not improve their measured 
functional independence over the period they had the tech- 
nology in their home. This could be attributed to a number 
of factors including the insensitivity of the chosen outcome 
measure, the specificity of the computer exercises and 
the number of repetitions carried out. Furthermore, it 
should be noted that all of the participants were > five 
months post-stroke and only used the system for up to 
five weeks. This may therefore not have been a long 
enough period of time to drive neuroplasticity and 
functional improvement [50]. 

CMOC refinement 

This research aimed to test and refine intervention the- 
ories by exploring the complex interactions of contexts, 
mechanisms and outcomes. This involves refining CMOC 
propositions which can then be retested (Figure 3). 

The combination of determining what works/does not 
work for whom in what circumstance and respects has 
identified elements of feedback that are essential for 
mechanisms to work. This has enabled the refinement 
of theories underpinning the use of computer-based 
feedback for post-stroke upper-limb rehabilitation in 
the home. Indeed, this research has made significant 



refinements to the proposed CMOC s. Two examples 
of these refinements are set out in Table 6. The full 
table of refined CMOC s are available as an Additional 
file 1. 

Discussion 

Existing literature suggests that concurrent feedback, 
knowledge of performance (KP), knowledge of results 
(KR) and explicit feedback may be key components in 
the promotion of improved performance. However, existing 
literature also highlighted the heterogeneity in studies that 
have explored the use of feedback in post-stroke rehabilita- 
tion [51]. This suggests that careful consideration should be 
made as to what form and method of delivery of feedback 
is given. In other words, the provision of feedback should 
not be a one size fits all' component of rehabilitation and 
different forms and methods of delivering feedback may be 
more effective at promoting self-managed rehabilitation in 
different contexts [51-53]. 

The findings have revealed that the mechanisms 
underpinning the utilisation of feedback from com- 
puter technology for upper-limb rehabilitation in the 
home are influenced by the pre-existing, personal and 
environmental contextual factors surrounding the user. 
The identification of the interaction between feedback, 
mechanisms and outcome(s) has also revealed how 
underpinning theories (theoretical components) may 
work in combination with other theories to produce an 
outcome within a given context. For example, knowledge 



Table 6 The Refined CMOC's 



CMOC's Plausible mechanisms: 'what' 



Contexts: 'for whom' and 'in what circumstances' 



Possible outcomes 



CMOC 1 M1: Receiving rewarding, KR feedback 

from the system might improve the user's 
confidence by confirming performance. 



CMOC 2 M2: By receiving KR and KP feedback, 
users might feel confident to be able 
to interpret their performance and 
changing their movements to improve 
subsequent performance(s). 



CI: A system that: 

• Is reliable, accurate and robust. 

• Can be adapted and personalised to the individual 
personal, environmental and social context of the 
stroke survivor 

• Is accessible in the home setting 

• Is used by the stroke survivor who does not believe 
they will become more socially isolated as a result 
of carrying out rehabilitation and subsequent ADL's, 

• Can be used independently of the therapist. 
C2: A system that: 

• Is reliable, accurate and 

• robust. 

• Can be adapted and personalised to the individual 
personal, environmental and social context of the 
stroke survivor 

• Can be used independently by the stroke survivor in 
the home. 



01: Adoption and development 
of a self-management approach to 
rehabilitation (behaviour change). 

- Independent rehabilitation, 
self-evaluation and self-monitoring 
of recovery. 



02: Development of 
self-management skills. 

- Problem-solving. 



• Is provided in an environment where the user 
is provided with adequate resources and support. 
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of results (feedback theory) [21] provided confirmation 
of performance (self-efficacy) [54] which motivated the 
user (Social Cognition Theory) [55], this gave the user the 
confidence (self-efficacy) [54] to use the system. This led 
to increased usage, repetitive use (motor-learning) [29,31] 
and increased their familiarity and use of the system 
(resource utilisation) [56]. In turn this repetitive use 
probably facilitated motor learning and improved results 
(motor learning) [29,31], reinforcement and increased 
motivation (Social Cognition Theory) [55]. This has 
highlighted the key ingredients that are necessary in 
activating the mechanisms underpinning the utilisation of 
computer-based feedback as well as the issues involved in 
testing early prototype technology and how developing 
technology systems need to be reliable, robust and accurate. 

This research has highlighted the need for computer 
feedback to be both accurate and reliable. Indeed, the 
accuracy (i.e. it would show the arm moving backwards 
when it is moving forwards) and reliability (i.e. the system 
would crash and/or not boot up correctly) of the equipment 
used in this study influenced the utilisation of feedback 
which in turn, hindered the mechanisms. Because the on- 
screen display was sometimes inaccurate, the participants 
(and in some instances the carers as well) became frus- 
trated, lost patience, and trust in the feedback provided. 
This resulted in the participants being less willing to use the 
equipment and/or dismissing the feedback as an inaccurate 
evaluation of their performance. Therefore, if similar sys- 
tems are to be used in the future for mainstream rehabilita- 
tion, accuracy and reliability is essential. 

However, despite the encouraging qualitative findings 
for the promotion of self-managed rehabilitation, the 
quantitative findings suggest that this did not result in 
functional improvement. Therefore, further work is re- 
quired to explore how overcoming technological and con- 
textual challenges may lead to the increased intensity and 
longer periods of use (months not weeks) which will drive 
neuroplastic adaptation [49]. 

Developing technology needs to account for the clin- 
ical needs of the practitioners as well as the end users 
themselves. If technology is to be used in mainstream 
therapy, service providers are required to consider the 
impact and variance of the context, the reliability and 
accuracy of the technology and importantly, what 
forms of feedback are provided to facilitate the users' 
understanding of performance. 

To have clinical utility, technology systems would need 
to incorporate greater clinical adaptability, such as, inclu- 
ding greater variety of exercises into the programme(s). 
For example, the PhysioTools (www.physiotools.com) 
[57] exercise sheets used currently (and by the therapists 
in this study) has over 16000 activities that can be selected 
and subsequently prescribed by the therapist. This enables 
therapists to choose from a vast library of activities what 



to prescribe making their therapy specific to the require- 
ments of the patient. Therefore, if therapists are to incor- 
porate technology into their practice, they may require 
and even expect the technology to be a better alternative 
to what is currently available to them. 

With the growing interest in gaming technology such 
as the Nintendo® Wii™; future users may be more receptive 
to computer interaction; however, they may also have 
higher expectations in terms of the interface and moti- 
vational components of rehabilitative devices. Therefore, if 
future systems are to be embraced they will have to meet 
the rising expectations of system users and therapists for 
both usability and clinical utility. 

Research also suggests that therapists currently provide a 
model of service delivery that is led by their empirical 
knowledge, which may not be conducive to motor learning 
and self-management. This may therefore limit the stroke 
survivor s ability to utilise computer-based feedback to 
facilitate self-managed rehabilitation. Work is required 
to educate therapists in the provision of therapy that in- 
cludes newer innovative methods of delivering feedback 
that facilitates independent rehabilitation. This will place 
less demand on services whilst empowering stroke survi- 
vors and close family members to carry out and continue 
their recovery beyond the acute and sub-acute period. 

Finally, the adoption of a realistic evaluation metho- 
dological approach used in this research gave an impor- 
tant innovative dimension to the study since this 
methodology has not been used in the development of 
technology for rehabilitation. In line with the MRC 
framework for complex interventions [58], this ap- 
proach has allowed for the refinement of the theories 
underpinning the intervention within the context of 
delivery and indeed tested the adoption of realistic 
evaluation for on-going research [23]. 

Further research is required to continue the cycle set 
out in Figure 1 where the refined CMOC s should be tested 
again with the refinements in place. This would involve 
changes to service delivery, therapy education, and the 
technology itself (as described previously). This would allow 
for further testing of the theoretical underpinnings of 
this intervention and the exclusion of the problematic 
non-conducive elements highlighted in this study. 

The SMART 2 project is currently exploring the imple- 
mentation and adoption of the key elements of feedback 
highlighted in this research through the development 
and user testing of the Personalised Self-Management 
Rehabilitation System for stroke (SMART 2) [23]. 

Conclusions 

This research has exposed the interaction between the 
feedback delivered through computer technology, the 
mechanisms underpinning the intervention, the context, 
and how this leads to variable outcome (s). 
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The identification of these interactions may therefore 
inform the development of technology; therapy educa- 
tion and the subsequent adoption of technology and a 
self-management paradigm; long-term self-managed 
rehabilitation; and importantly, improvements in the 
physical and psychosocial aspects of recovery. Further 
work is required to; develop technology so that it incorpo- 
rates the elements of feedback highlighted by this research; 
ensure the technology is robust, reliable and accurate; 
investigate the clinical utility of technology for home- 
based stroke rehabilitation, and the extent to which it 
might encourage utilisation by the end user. 

Additional file 



Additional file 1: Additional Material: Refined CMOCs. 
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